AD-A113  768 

UNCLASSIFIED 


HONEYWELL  SYSTEMS  AND  RESEARCH  CENTER  MINNEAPOLIS  MN  F/6  S/9 

TEAM  TRAININ6  FOR  COMMAND  AND  CONTROL  SYSTEMS:  RECOMMENDATIONS  —ETC(U) 
APR  02  DR  BAUM*  J  A  MODRICK  F3361S-79-C-0025 

AFHRL-TP-B2-9  NL 


ALE  copy  AD  AH  37 68 


AFHRL-TP-82-9 


AIR  FORCE  m 

nH 

U 
M 
A 
N 


TEAM  TRAINING  FOR  COMMAND  AND  CONTROL  SYSTEMS. 
RECOMMENDATIONS  FOR  APPLICATION 
OF  CURRENT  TECHNOLOGY 


David  R.  Baum 
John  A.  Modrick 
Sam  B.  Hollingsworth 

Honeywell  Systems  and  Kesearch  Center 
2600  Ridge»av  Parkway 
P.O.  Box  312 

Minneapolis.  Minnesota  53440 


LOGISTICS  AM)  TECHNICAL  TRAINING  DIVISION 
Logistics  Research  Branch 
Wright-Patterson  Air  Force  Base,  Ohio  45433 


April  1982 

Final  Technical  Paper 


Approved  for  public  release:  distribution  unlimited. 


LABORATORY 


AIR  FORCE  SYSTEMS  COMMAND 

BROOKS  AIR  FORCE  BASEJEXAS  78235 


04  23  086 


NOTICK 


VI  lirn  Government  drawings.  specifications.  or  oilier  dala  are  used  for  anv  purjmse 
oilier  llian  in  eon  Heel  ion  will)  a  definite!)  Government-related  procurement.  the  foiled 
Slate-  Government  incurs  no  responsibility  or  anv  obligation  whatsoever.  The  fact  that 
the  Oovernment  may  have  formulated  or  in  anv  wav  supplied  the  said  drawings, 
specifications,  or  oilier  data,  is  not  to  be  regarded  bv  implication,  or  otherwise  in  anv 
manner  construed,  as  licensing  the  bolder,  or  anv  other  person  or  corporation:  or  as 
conveying  any  rights  or  permission  to  manufacture,  use.  or  sell  anv  patented  invention 
that  may  in  any  way  be  related  thereto. 

The  Public  Vffairs  Office  I  las  reviewed  this  paper,  and  it  is  releasable  to  the  National 
lechnical  Information  Service,  where  it  will  be  available  to  the  general  public, 
including  foreign  nationals. 

This  paper  has  been  reviewed  and  is  approved  for  publication. 

ROLAND  VC.  DENSON 
Contract  Monitor 

ROSS  L.  MORGAN.  Technical  Director 
Logistics  and  Technical  Training  Division 


DONALD  C.  TF.TMEYER.  Colonel.  I  SAF 
Chief.  Logistics  and  Technical  Training  Division 


I'nrlassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (When  Dat  Entered). 

REPORT  DOCUMENTATION  PAGE 

READ  INSTRUCTIONS 

BEFORE  COMPLETING  FORM 

1.  REPOST  NUMBER  2.  GOVT  ACCESSION  NO. 

76X 

3  RECIPIENT’S  CATALOG  NUMBER 

4.  TlTuE  (and  Subtitle) 

I  F. AM  TRAINING  FOR  COMMAND  AND  CONTROL  SYSTEMS: 
RECOMMENDATIONS  FOR  APPLICATION 

OF  Cl'RRENT  TECHNOLOGY 

S.  TYPE  OF  REPORT  &  PERIOD  COVERED 

Final 

S.  PERFORMING  ORG.  REPORT  NUMBER 

7.  AUTHORfsJ 

David  R.  Baum 

John  A.  Modriek 

Sam  R.  Hollingsworth 

8.  CONTRACT  OR  GRANT  NUMBERf.J 

F3361 5-79-C-0025 

9.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

Honeywell  Systems  and  Research  Center 

2000  Ridgeway  Parkway.  P.0.  Box  312 

Minneapolis.  Minnesota  55140 

riO.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMBERS 

62205F 

17100347 

It.  CONTROLLING  OFFICE  NAME  ANO  ADDRESS 

lit,)  Air  Force  Human  Resources  Laboratory  (AFSC) 

Brooks  Air  Force  Base.  Texas  70235 

12.  REPORT  DATE 

April  1982 

>3.  NUMBER  OF  PAGES 

02 

14.  MONITORING  AGENCY  NAME  A  ADDRESSfif  dilleren  1  from  Controlling  Oftice) 

Logistics  and  Technical  Training  Division 

Air  Force  Human  Resources  Laboratorv 

Wright-Patterson  Air  Foree  Base.  Ohio  45433 

IS.  SECURITY  CLASS,  (ot  thlt  report) 

Unclassified 

ts  a.  DECLASSIFICATION/ DOWNGRADING 
SCHEDULE 

16.  DISTRIBUTION  STATEMENT  (of  thte  Report) 

Approved  for  public  release:  distribution  unlimited. 

17.  DISTRIBUTION  STATEMENT  (of  the  abstract  entered  in  Block  20,  if  different  from  Report) 


is.  supplementary  notes 


19.  KEY  WORDS  (Continue  on  reverse  side  it  necessary  and  identify  by  block  number) 

eommand  and  control 
Tactical  Air  Command 
team  training 

20.  ABSTRACT  (Continue  on  reverse  side  If  necessary  and  Identify  by  block  number) 

This  paper  contains  recommendations  for  solutions  to  problems  in  team  training  IT*)  in  command  and  control 
1C*)  systems  in  the  Air  Force.  The  recommended  solutions  are  proposed  as  projects  within  the  current  stale  of 
technology.  They  are  near-term  projects  and  require  only  design  and  preparation  for  an  existing  application  rather 
than  further  development  or  research.  The  recommended  projects  were  generated  to  address  issues  and  problem  areas 
identified  in  a  survey  of  AFC*T*. 

*  t 

Eleven  projects  were  recommended,  consisting  of  taxonomy  development,  methodology  for  team  task  analysis, 
evaluation  of  team  readiness,  guidelines  and  aids  for  defining  simulated  combat  mission,  formal  training  materia)  for 

DD  ,  1473  E0.t,on  of  t  NOV  SS  IS  OBSOLETE  I  nrhlssifird 


t  uclassified 

SECURITY  CLASSIFICATION  OF  THIS  P  AGEftrhsn  D*f  Enfrtd)  _ _ 

Item  20  (Continued): 

lj  system  supervisory  personnel,  development  of  siipplemenlal  skill  training  for  entrant*  into  the  air  weapons 
controller  career  field,  formal  training  for  operators  of  interceptor  simulators,  procedures  for  program  evaluation, 
procedures  tor  compiling  and  using  after-action  reports,  attitude  survey  of  discontent  in  ( career  fields,  and  an 
instructor  resource  package  lor  operational  unit  instructors.  The  plan  for  a  project  to  implement  a  technique  for 
training  is  discussed  in  greater  detail.  <j, 

Four  additional  technical  papers  are  concerned  with  \I  C2T2  research:  AFHRI.-TP-82-7.  \FHRI.-TP-82-8. 
\FHHI -  IT-82-III.  and  \FHRI.-TP-82-l  1. 


AFHRL  Technical  Paper  82-9 


April  1982 


TEAM  TRAINING  FOR  COMMAND  AND  CONTROL  SYSTEMS: 
RECOMMENDATIONS  FOR  APPLICATION  OF  CL'RRENT  TECHNOLOGY 


By 

David  R.  Baum 
John  A.  Modrick 
Sam  R.  Hollingsworth 

Honeywell  Systems  and  Research  Center 
2600  Ridgeway  Parkway 
P.O.  Box  312 

Minneapolis,  Minnesota  55440 


Reviewed  by 

Roland  W.  Denson 
Research  Psychologist 
Ground  Operations  Section 


Submitted  for  Publication  by 


William  B.  Askren 
Senior  Scientist 
Logistics  Research  Branch 


PREFACE 


I  hroughoul  ill*’  li'Xl  of  lliis  paper.  reference  is  made  lo  volumes  I  through  V.  Tliese  volumes  have  been 
published  as  so  pa rale  leehnieal  papers  identified  as  follows: 

\  olume  I 

Baum.  D.R..  Modriek.  J.  A..  X  Hollingsworth.  S.R.  Team  training  for  command  and  control  systems: 
Status.  AFIIRI.-TI’-Hio,  %  rigltl-l’atterson  AFB.  OH:  Logistics  am)  Teehnieal  Training  Division.  \ir 
Force  Human  Resourees  Laboratory.  April  1  *>82. 

Volume  II 

Modriek.  J. A..  Baum.  D.H..  &  Hollingsworth.  S.R.  Team  training  ( or  command  and  control  systems: 
Recommendations  Jor  research  program.  AFHRL-TP-82-8.  Wright-Paltersoll  AFB.  OH:  Logistics  and 
leehnieal  Training  Division.  Air  Force  Human  Resourees  Lahoralorv.  April  1982. 

Volume  III 

Baum.  D.R..  Modriek.  .I.A..  X  Hollingsworth.  S  R.  Team  training  for  command  and  control  systems: 
Recommendations  for  application  of  current  technology.  AFHRL-TP-82-9.  Wrigitl-Pallerson  AFB. 
OH:  logistics  and  Technical  Training  Division.  Air  Force  Human  Resources  Lahoralorv.  April  I *#82. 

Volume  IV 

Hollingsworth.  S.R..  Modriek.  J.A..  &  Baum.  D.R.  Team  training  for  command  and  control  systems: 
Recommendations  for  simulation  facility.  AFHRL-TP-82-IO.  Wrighl-Patlerson  AFB.  OH:  Logistics 
and  Technical  Training  Division.  Air  Force  Human  Resources  Laboratory.  April  1982. 

Volume  V 

Baum.  D.R..  Modriek.  J.  Hollingsworth.  S.R.  Team  training  for  command  and  control  systems: 
l:\es  ulite  summon.  A  FURL -TP-82-1  i.  Vt  right-Patlcrson  AFB.  OH:  Logistics  and  Technical  Training 
Division.  Air  Force  Human  Resourees  Laboratory.  April  1982. 

This  paper  is  the  third  of  live  volume-  prepared  by  Honeywell  to  document  the  results  of  a  research 
program  lo  evaluate  the  current  statu-  ol  team  training  (T2)  for  operators  of  complex  Air  Force  Command 
and  Control  (AH.2)  systems,  and  to  make  recommendations  for  enhancing  the  AFC*T*  process.  The  research 
wa-  performed  for  the  Air  Force  Human  Resources  Laboratory  under  contract  F33615-79-C-0025. 

The  pur|Mise  of  the  present  volume  is  lo  describe  AFF2T2  problem  areas  amenable  lo  solution  through 
the  application  of  technology  that  is  currently  available.  The  description  of  each  problem  area  includes  a 
summary  of  the  issue,  a  description  of  sources  of  information  to  be  used  in  the  solution,  the  general  nature  of 
the  -olulion.  its  impact,  and  estimates  of  contractor  and  Air  Force  resources  necessarv  for  its  implementation. 

I  his  research  effort  supports  a  major  new  Air  Fort—  Human  Resources  Laboratory  (AFHRL)  research 
and  development  program  whose  primary  objective  is  lo  improve  T2  technologies  in  areas  parlieularlv 
relevant  to  Air  Force  combat  readiness.  The  program  objective  requires  the  establishment  of  a  baseline  data 
base  on  how  T2  is  currently  conducted  in  the  Air  Force:  how  it  is  developed,  implemented,  and  evaluated. 
Bc'-atjsc  Air  Force  teams  vary  greatly  in  size,  structure  and  functions,  it  would  be  impractical  to  collect  data 
on  the  training  provided  lo  all  of  them.  Rather,  the  scope  of  this  research  effort  had  lo  be  directed  at  an  area 
with  potential  high  payoff  for  increased  combat  readiness  and  effectiveness.  The  area  of  command  and 
control  (F2)  was  chosen  as  a  point  of  departure  for  the  research  because  C2  teams  lend  lo  be  well  defined 
structurally,  are  of  a  manageable  size,  and  perform  functions  highl\  representative  of  Air  Force  mission 
needs.  Furthermore,  as  the  research  effort  unfolded,  limited  lime  and  resources  made  it  necessarv  lo  focus  on 
tactu  al  and  air  defense  F2  sy  -lent-  to  the  exclusion  of  strategic  C.2  s\ stems.  Thus,  the  F2  sy  stems  surveyed  are. 
or  to  the  ea-e  of  planned  -y stems  yyill  become.  Tactical  Air  Fommaml  (T.AF)  resources. 


Tile  goal  of  this  effort  was  to  develop  a  picture,  through  interview  and  observation,  of  how  AFC*T*  is 
currently  developed,  implemented,  and  evaluated,  and  what  C2  training  needs  will  arise  in  the  future.  Rased 
on  this  picture,  strengths  and  weaknesses  of  AFC'2T2  were  identified,  and  recommendations  were  developed 
in  three  areas: 


•  T2  research  and  development  program 

•  Resolution  of  issues  using  current  leehni<pies/leelinologies 

•  Simulation  lechnologv  development  for  f.'2T2 

1  hese  recommendations  will  form  the  foundation  for  future  research  bv  AFHRI.  into  the  performance 
of  (,2  teams  and  systems.  The  research  will  encompass  training  teclinologv.  performance  measurement 
technii|ues  for  C2  teams  and  systems,  human  resources  issues  in  the  design  and  operation  of  C2  svslems.  and 
training  ol  commarid/derision  skills.  The  ultimate  goal  of  this  program  is  to  improve  technologies  in  areas  of 
team  and  human  factors  related  to  the  combat  effectiveness  of  Air  Force  C2  operations. 
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CHAPTER  I 


INTRODUCTION  AND  OVERVIEW 


The  objective  of  this  study  was  to  survey  and  characterize  the  status  of 
2  2 

AFC  T  programs.  The  methods  used  to  define,  develop,  impiement,  and 

2 

evaluate  tactical  air  C  training  programs  were  described,  analyzed,  and 

2  2 

evaluated.  The  evaluation  permitted  identification  of  AFC  T  strengths 
and  weaknesses.  The  weaknesses  were  consolidated  into  issue  areas 
according  to  whether  they  involved  training  program  definition  and  develop¬ 
ment,  implementation,  evaluation  and  modification,  or  personnel  policy 
and  resource  constraints.  The  issues  are  summarized  in  Table  1. 

2  2 

The  purpose  of  the  present  volume  is  to  describe  those  C  T  problem 

areas  amenable  to  solution  through  the  application  of  currently  available 

or  refinable  technology.  The  emphasis  in  the  present  volume  is  on 

identifying  existing  or  potential  training  technology  which  could  provide 

2  2 

near-term  improvements  in  the  cost-effectiveness  of  AFC  T  programs. 
The  description  of  each  problem  area  includes  a  summary  of  the  issue,  the 
specific  sources  of  information  that  could  be  used  in  the  solution,  the 
general  nature  of  the  anticipated  solution,  its  impact,  estimates  of 
contractor  resources  necessary  for  full  development,  and  estimates  of 
Air  Force  resources  necessary  for  implementation. 

Each  proposed  solution  is  further  evaluated  as  to  appropriateness  for 
inclusion  in  a  user-oriented  source /reference  book  (SRB).  The  SRB  would 
present  procedures  to  be  used  by  training  developers  and  managers  to 
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TABLE  1.  AFC  T  ISSUES  AND  PROBLEM  AREAS 
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develop  training  materials  for  C  team  supervisors.  The  SRB  would 
describe  procedures  for  developing  a  job  task  list,  identifying  skill 
categories,  establishing  skill  acquisition  guidelines,  defining  experience 
exercise  requirements,  selecting  appropriate  media,  and  developing 
scripted  exercises.  The  criteria  by  which  solutions  are  judged  appropriate 
for  an  SRB  are  fully  documented.  These  criteria  include  problem  area 
frequency  and  criticality,  solution  generality,  intended  users,  and  ease 
of  solution  application  by  the  user. 

The  remainder  of  this  volume  is  organized  into  three  chapters.  The 
description  and  evaluation  of  issues  are  presented  in  Chapter  II.  The  topic 
selected  for  application  in  Phase  II  of  this  effort  is  described  in  fuller 
detail  in  Chapter  III.  The  material  to  be  included  in  the  SRB  is  presented 
in  outline  format  in  Chapter  IV.  The  outline  has  been  annotated  to  indicate 
the  type  or  depth  of  coverage  each  topic  should  receive. 


3 


chapter  II 


DESCRIPTION  AND  EVALUATION  OF  AFC2T2  ISSUES  AMENABLE 
TO  SOLUTION  THROUGH  EXISTING  TECHNOLOGY 

2  2 

This  chapter  presents  those  C  T  issues  judged  amenable  to  solution 
through  the  application  or  refinement  of  existing  training  technology. 

Each  issue  is  described  in  terms  of  the  approach,  impact,  and  cost  of 
the  proposed  solution.  The  appropriateness  of  including  each  issue  in  an 
SRB  is  also  evaluated. 

The  statement  of  approach  and  resources  required  for  each  issue  is  stated 
more  globally  in  the  present  volume  than  is  ultimately  needed  for  planning 
and  implementing  the  project.  However,  a  more  detailed  statement  requires 
at  least  one  more  iteration  of  the  recommendations.  That  iteration  would 
provide  more  detailed  information  and  better  definition  of  the  specific 
topic,  materials  to  be  designed  and  prepared,  and  the  research  methodology 
and  procedure  to  be  followed.  At  that  point,  person-year  requirements  and 
budget  could  be  estimated.  Such  information  is  not  available  at  present, 
nor  are  there  resources  to  support  the  analysis.  The  present  level  of 
discussion  is  intended  to  identify  research  objectives  and  strategies,  which 
may  then  become  the  basis  for  more  detailed  planning. 

Table  2  summarizes  the  specific  close-in  applications  discussed  in  this 
chapter.  The  issues  listed  in  Table  1  have  been  restated  as  solutions  or 
development  approaches.  Some  of  the  issues  listed  in  Table  1  are  much 
more  general  than  others;  in  these  instances,  the  proposed  specific  need 


TABLE  2.  IMMEDIATE  APPLICATIONS  FOR  RESOLVING  C2T2  ISSUES 
Definition  and  Development 

•  Taxonomy  for  identifying  and  classifying  team-oriented 
tasks  and  skills 

•  Methodology  for  analyzing  team-oriented  tasks 

•  Criteria  for  evaluating  team  readiness 
Implementation 

•  Guidelines  and  aids  for  defining  simulated  combat 
mission  training  objectives  and  characteristics 

2 

•  Formal  training  material  for  selected  C  system/team 
supervisory  personnel 

••  Supplemental  training  in  selected  skills  for  entrants  into 
the  Air  Weapons  Controller  career  field 

•  Formal  training  for  intercept  pilot  simulators 
Evaluation 

2 

•  Improved  validity  of  T  program  evaluation  questionnaires 

•  Procedure  for  compiling  after-action  reports  into  a 
training  exercise  data  base 

Personnel  Policy 

2 

•  Attitude  surveys  to  pinpoint  sources  of  discontent  in  C“ 
career  fields  and  recommend  changes 

•  Instructor  resource  package  for  operational  unit  instructors 
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deals  with  an  initial  step  that  can  be  made  using  current  technology.  Some 

general  issues  have  spawned  more  than  one  development  need.  For 

example,  techniques  exist  for  a  first  step  to  develop  a  conceptual  framework 

for  identifying  and  analyzing  team  skills;  the  proposed  approach  to  develop- 

2 

ing  training  materials  for  C  team  supervisors  can  provide  a  relatively 
complete  solution  to  the  problem  of  a  lack  of  formal  training  for  those 
personnel. 

T2  DEFINITION  AND  DEVELOPMENT 

2 

A  fundamental  problem  in  T  definition  is  that  current  Instructional 
System  Development  (ISD)  guidelines  and  procedures  do  not  address  team- 
oriented  tasks  and  skills.  Rather,  ISD  focuses  on  an  operator's  individual 
knowledge  and  skill  requirements,  especially  as  related  to  structured  tasks 
with  equipment-oriented  procedures.  This  focus  is  necessary  but  is  not 
sufficient  to  deal  with  task  performance  involving  interpersonal  inter¬ 
action  and  requiring  cognitive,  team-oriented  skills. 

The  major  goal  of  the  training  requirements  analysis  (TRA;  sometimes 
called  front-end  analysis)  of  the  ISD  procedure  is  the  specification  of 
training  objectives.  The  steps  in  this  analysis  have  been  described 
elsewhere  in  this  report  (Volume  I,  Chapter  II).  They  are: 

1.  Identify  job  requirements 

2.  Select  tasks  to  be  trained 

3.  Analyze  selected  tasks 

4.  Construct  job  performance  measures 

5.  Develop  training  objectives 
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Other  frameworks  exist  for  systematically  carrying  out  a  TRA.  For 
example,  Smode  (Reference  1)  presented  a  five-step  process  which  is  at 
a  more  basic  level  than  that  outlined  above.  A  summary  of  the  procedures 
can  be  found  in  Hall  and  Rizzo  (Reference  2)  or  Denson  (Reference  3). 

1.  Describe  and  analyze  operational  system 

2.  Define  the  task  structure 

3.  Analyze  tasks 

4.  Prepare  detailed  task  statements 

5.  Express  task  statements  in  the  form  of  training  objectives 

These  two  sets  of  steps  are  by  no  means  incompatible.  The  identification 
of  job  requirements,  for  example,  is  based  on  an  analysis  of  the  operational 
system  and  definition  of  the  task  structure.  A  good  approach  would  be  to 
merge  the  procedures,  ensuring  that  the  level  of  detail  implied  in  Smode' s 
process  is  not  lost  in  the  merger. 

It  is  not  enough,  however,  simply  to  present  the  steps  of  a  procedure  and 
describe  what  is  to  be  done.  The  conceptual  framework  determines  the 
nature  of  the  output  which  accompanies  the  implementation  of  the  steps. 
Current  frameworks  do  not  possess,  for  example,  a  classification  scheme 
which  allows  identification  of  team-oriented  tasks.  Nor  do  they  have  a 
methodology  for  the  analysis  of  tasks  requiring  team-oriented  knowledges 
and  skills,  or  comprehensive  techniques  for  evaluating  team  performance 
and  readiness.  The  development  of  such  tools  is  within  the  current  state 
of  the  art  in  training  technology. 
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There  are  three  interrelated  efforts  which  can  be  undertaken  to  develop  and 
apply  available  techniques  to  overcome  these  shortcomings  of  the  TRA: 

1.  Develop  taxonomy  for  identifying  and  classifying  team-oriented 
tasks  and  skills. 

2.  Develop  methodology  for  analyzing  team-oriented  tasks. 

3.  Develop  criteria  for  the  evaluation  of  team  readiness. 

2 

The  first  two  efforts  are  focused  on  the  definition  of  T  requirements  for 
the  individual.  The  third  effort  is  aimed  at  the  team  as  a  whole. 


Taxonomy  Development 


Examples  of  task  taxonomies  abound  in  the  literature  of  training  and  human 
factors  (References  4  through  8).  The  extant  taxonomies  are  characterized 
by  having  been  developed  for  fairly  narrow  applications  or  with  a  particular 
orientation  to  taxonomic  issues  and  purposes.  Of  pertinence  to  the  present 
case  is  that  all  of  these  taxonomies  deal  with  tasks  performed  by  a  single 
operator.  Despite  this  fact,  the  existing  behavior  taxonomies  are  a 
primary  source  of  information  which  can  be  used  to  develop  a  taxonomy 
oriented  towards  describing  and  classifying  team  behaviors. 


A  second  important  source  of  information  is  the  operational  team  environ¬ 
ment.  Observations  of  actual  teams  carrying  out  (simulated)  combat 
missions  must  be  undertaken  to  develop  and  validate  the  taxonomy.  To 
achieve  representativeness  and  generality,  the  Tactical  Air  Control 
System  (TACS),  Tactical  Air  Control  Center  (TACC),  or  Control  and 
Reporting  Center/ Post  (CRC/P)  should  provide  the  observation  environment. 
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Initial  observations  would  be  carried  out  with  a  taxonomy  developed  with  a 
cognitive  skill  orientation;  data  descriptive  of  task  frequency  over  mission 
segments  would  be  collected. 

The  result  of  this  ethological  study  of  teams  will  be  a  behavioral  taxonomy 

which  forces  attention  to  team- oriented  tasks  in  the  initial  steps  of  the 

ISD  procedure.  This  revision  of  the  ISD  procedure  will  permit  accommoda- 

2 

tion  of  the  definition  of  T  requirements  and  objectives  in  a  systematic, 
verifiable  fashion. 

This  is  a  difficult  and  complex  task.  It  is  estimated  that  full  development 

of  the  taxonomy  would  require  the  following  contractor  resources:  Two 

and  one-half  person-years  of  effort  would  be  necessary,  divided  among 

a  full-time  senior  and  a  junior  psychologist,  both  of  whom  are  knowledge- 
2  2 

able  in  the  C  T  area,  and  a  half-time  research  associate.  Field 
observations  would  require  extensive  cooperation  and  understanding  of 
participating  units  and  a  substantial  amount  of  travel  on  the  part  of  the 
researchers.  The  time  course  of  the  study  would  be  determined  primarily 
by  the  frequency  of  training  exercises,  but  a  minimum  of  three  observations 
would  be  necessary.  Required  resources  include  a  data  recorder  and 
access  to  computer  facilities  for  data  reduction.  Gunning  and  Manning 
(Reference  9)  discuss  the  use  of  this  strategy  in  the  context  of  an  aircrew 
task  loading  study. 

The  implementation  of  the  taxonomy  as  a  front  end  to  the  ISD  procedure 
will  require  no  substantial  Air  Force  resources  beyond  publishing  a 
revision  of  the  Air  Force  ISD  model  to  be  used  by  all  training  developers. 

It  is  important  to  note,  however,  that  the  use  of  a  revised  ISD  procedure 
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with  a  team  behavior  orientation  can  be  expected  to  result  in  more  effective 

but,  perhaps,  more  expensive  training.  The  ultimate  cost  (or  savings) 

2 

will  depend  on  the  specific  T  objectives  that  result  and  how  they  are 
sequenced  throughout  existing  training  programs. 

The  taxonomy  developed  through  this  effort  would  be  appropriate  for 

_  2 
inclusion  in  an  SRB.  The  problem  addressed  is  common  to  all  T  ,  as  is 

the  proposed  solution.  Furthermore,  according  to  the  posture  adopted 

herein  a  team  behavior  taxonomy  is  the  first  step  in  the  development  of  a 

conceptual  framework  embracing  teams  and  team-oriented  skills,  the 

2 

missing  fundamental  element  in  T  definition  and  development. 

Develop  Methodology  for  Team  Task  Analysis 

Identifying  and  classifying  team-oriented  tasks  is  the  first  step  necessary 
2 

in  defining  T  objectives.  Task  analysis  is  the  second  step.  Current  task 
analysis  methods  provide  for  analysis  of  individual  operator  tasks.  These 
methods  must  be  upgraded  to  support  the  analysis  of  team-oriented  tasks. 

The  purpose  and  results  of  a  task  analysis  are  discussed  in  Chapter  II  of 
Volume  I.  Briefly,  conditions  and  standards  of  task  performance  are 
determined.  The  results  are  a  detailed  description  of  tasks  and  task  steps, 
and  associated  environmental  and  equipment  conditions  and  behavioral 
criteria  of  performance. 


Perhaps  the  most  significant  shortcoming  of  current  task  analysis  methods 
is  that  they  are  primarily  equipment-oriented.  It  is  infrequently  acknowledged 
that  environmental  conditions  often  include  a  team  environment.  The 


performance  of  certain  tasks  will  be  contingent  upon  receiving  or  trans¬ 
mitting,  either  verbally  or  machine- assisted,  information  from  or  to 
another  team  member;  or  tasks  will  be  coordinated  in  sequencing  and 
timing  with  other  team  members  or  elements.  Take,  for  example,  data 
item  description  (DID)  DI-H-6130,  Task  and  Skill  Analysis  Report 
(Reference  10). 

Table  3  presents  the  DID  instructions  for  producing  a  description  and 
analysis  of  operator  tasks.  The  data  format  is  a  matrix,  as  shown  in 
Figure  1.  Clearly,  there  is  little  room  for  the  inclusion  of  team-oriented 
tasks  and  skills  in  the  analysis.  Importantly,  task  analysis  techniques  in 
various  ISD  procedures  are  essentially  the  same  as  that  in  DI-H-6130. 

Based  upon  the  data  obtained  in  developing  a  team-behavior  taxonomy,  the 

ISD  and  DI-H-6130  task  analysis  formats  should  be  expanded  to  produce 

detailed  descriptions  of  tasks  involving  interpersonal  dependencies  and 

interaction.  Such  descriptions  should  be  from  a  cognitive  skill  perspective. 

Determining  the  conditions  and  standards  of  the  performance  will  require 

2 

subject  matter  expertise  related  more  to  efficient  and  effective  C  team 
performance  than  to  the  specific  system  which  is  the  subject  of  analysis. 
That  is,  the  expert  should  be  a  generalist  of  supervisory  rank  rather  than 
a  technical  specialist. 

It  is  difficult  to  estimate  the  contractor  resources  necessary  to  fully  develop 
a  revised  ISD  task  analysis  or  DID  for  a  Task  and  Skill  Analysis  Report;  it 
is  an  ambitious  undertaking  and  is  at  least  partially  contingent  upon  the 
success  of  the  taxonomy  development.  If  the  same  contractor  were  involved 
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TABLE  3.  INSTRUCTIONS  FOR  CARRYING  OUT  OPERATOR 
TASK  ANALYSIS  (from  DI-H-6130) 


EXHIBIT  1 

OPERATOR  TASK  DESCRIPTIONS  AND  ANALYSIS 


Exhibit  1  identifies  and  provides  the  specific  operator  tasks  and  related 
equipment  required  to  operate  and  support  the  aeronautical  system/subsystem/ 
equipment. 


Instructions  for  Completing  Exhibit  1 


Block  No. 


Instructions 


1  Nomenclature--Enter  nomenclature,  part  number,  etc. 
Date~~Enter  date  of  preparation/ revision. 

2  Task  Identification- -Make  an  alphanumerical  listing  of 
tasks  which  will  provide  easy  identification  and 
correlate  with  maintenance  engineering  analysis  data 
where  applicable.  Tasks  shall  be  grouped  by  subsystem 
and  in  chronological  order. 

3  Task  Description-- Enter  the  complete  task  description. 

4  Step  Number--Enter  the  numerical  listing  of  each  task 
step. 

5  Step  Description-- Enter  the  logical  sequential  steps  for 
task  performance.  A  description  of  any  human  factor 
considerations  such  as  environmental  conditions,  speed 
of  action,  perception,  coordination,  physical  dexterity, 
safety,  etc.,  shall  be  included.  A  description  of  system/ 
equipment  condition  which  is  necessary  prior  to  the  task 
accomplishment  shall  be  included. 
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Equipment-- Enter  a  listing  of  equipment,  tools,  test 
equipment  required  for  the  specific  task  step. 


Figure  1.  Format  for  Operator  Task  Description  and  Analysis  (from  DI-II-6130) 


in  the  two  developments,  a  great  economy  could  be  achieved,  tinder  these 
conditions  the  level  of  effort  required  would  be  1  1/2  person-years  divided 
among  a  quarter-time  senior  scientist,  a  quarter-time  junior  scientist,  and 
a  full-time  research  associate.  The  cooperation  and  participation  of 
knowledgeable  Air  Force  personnel  would  be  essential.  The  Tactical  Air 
Control  System  Office  of  Training  Development  (TACS/OTD)  could  provide 
the  required  expertise  and  test  bed  for  development  and  implementation.  The 
duration  of  this  effort  would  be  on  the  order  of  six  months. 

Air  Force  resources  required  for  the  implementation  of  a  revised  ISD  or 

DI-H-6130  task  analysis  would  not  be  substantial.  The  impact  would  be 

improved  identification  of  team  performance  requirements  and  training 

objectives.  However,  as  with  the  team  behavior  taxonomy,  it  is  difficult 

2 

to  predict  the  ultimate  cost  or  savings  of  T  based  on  systematically- derived 
T2  objectives. 

The  revised  task  analysis  methodology  developed  through  this  effort  would 
be  appropriate  for  inclusion  in  an  SRB.  The  same  criteria  apply  here  as 
in  the  taxonomy  development.  The  problem  addressed  is  both  general  and 
critical,  and  the  proposed  solution  will  be  usable  by  the  intended  users 
because  it  is  an  extension  of  an  existing  method. 

Develop  Criteria  for  Evaluating  Team  Readiness 

What  are  the  dimensions  of  team  performance  to  which  an  expert  evaluator 
attends?  What  criteria  does  an  expert  apply  to  judge  the  combat  readiness 
of  a  team?  These  questions  are  candidates  for  development  involving  the 
application  of  multiattribute  utility  theory  (MAUT)  to  team  evaluation 
de  c  ision-  ma  king. 


At  present  the  evaluative  techniques  for  determining  the  readiness  of  teams 
are  either  objective  and  uninformative  or  too  subjective  to  be  of  use  in 
training  effectiveness  assessment.  Number  of  exercises  participated  in  and 
staffing  level  are  examples  of  objective  measures  with  little  information. 

On  the  other  hand,  the  format  of  narrative  reports  such  as  the  after-action 
report  (AAR),  while  providing  a  source  of  useful  information  regarding 
team  performance,  tends  to  be  unsystematic  and  loosely  structured.  A 
solution  to  this  problem  lies  in  the  development  of  an  evaluative  tool  which 
structures  and  externalizes  expert  judgment.  MAUT  provides  the  means  of 
developing  an  evaluative  tool  with  those  desirable  characteristics. 

The  approach  of  MAUT  consists  of  extracting  from  experts  the  independent 

dimensions  which  they  consider  important  in  a  particular  decision  context 

and  then  obtaining  assessments  of  the  value  or  utility  of  particular  attributes 

2 

on  the  dimensions.  For  example,  in  the  tactical  air  C  area  a  dimension  of 
readiness  might  be  the  timely  transmission  of  target  status  information  from 
the  surveillance  section  to  the  weapons  section.  Particular  attributes  might 
be  reflected  by  an  ail-or-none  outcome,  for  example,  target  data  trans¬ 
mitted,  or  a  utility  contingent  upon  the  delay  transpiring  between  a  change 
in  target  status  and  data  transmission. 

The  benefits  of  MAUT  have  been  successfully  demonstrated  in  the  context 

of  Marine  Corps  readiness  evaluation  (Reference  11);  but  this  application 

was  aimed  at  a  higher  level  of  detail  than  that  required  to  solve  the  present 

problem.  Nevertheless,  the  MAUT  decision  analytic  framework  should  be 

2 

readily  applicable  to  the  evaluation  of  C  team  readiness.  The  development 

of  a  systematic,  highly- structured  measurement  tool  is  essential  to  the 

2  2 

accomplishment  of  effective  T  and  valid  T  research. 


2 

Contractor  resources  required  to  fully  develop  a  technique  for  C  readiness 

assessment  are  estimated  at  between  four  and  five  person-years 

over  a  12  to  18-month  period.  The  person-years  should  be  distributed 

2  2 

equally  among  a  senior  psychologist  with  expertise  in  the  C  T  area,  two 
decision  analysts  with  expertise  in  applying  MAUT,  and  an  experienced 
research  assistant.  Extensive  involvement  of  Air  Force  standardization/ 
evaluation  personnel  would  be  necessary.  A  representative  and  general 
solution  could  be  achieved  by  using  an  element  of  the  TACS  as  the  object 
of  the  development. 

Implementation  of  a  fully-developed  technique  for  readiness  assessment 
would  require  training  of  ail  C  standardization /evaluation  personnel  and 
the  institution  of  a  management  information  data  base  for  organizing  the 
evaluative  data  and  procedures  for  using  it. 

This  development  of  the  technique  as  a  topic  is  not  appropriate  for  inclusion 
in  an  SRB.  However,  the  implementation  of  a  technique  for  readiness 
assessment  would  be  best  accomplished  within  the  SRB  framework. 

T2  IMPLEMENTATION 

Develop  Guidelines  and  Aids  for  Defining  Simulated 
Combat  Mission  Training  Objectives  and  Characteristics 

Simulated  combat  missions  (SCMs)  or  system  training  exercises  (STEs)  are 

2 

the  primary  vehicles  for  accomplishing  T  in  operational  units.  The 

2 

overriding  requirement  for  SCMs/STEs  is  that  C  teams  experience  the 
quality  and  quantity  of  events  and  activities  which  might  occur  in  actual 
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combat.  In  order  to  specify  the  characteristics  of  SCMs/STEs,  working 
group  conferences  meet  on  a  regular  (quarterly)  basis  to  plan  them.  The 
planning  procedure  is  ad  hoc  and  is  usually  constrained  by  limited  live 
resources.  There  is  currently  no  set  of  guidelines  or  aids  for  defining 
training  objectives  or  for  specifying  the  characteristics  of  SCMs/STEs 
necessary  to  meet  the  objectives.  In  fact,  the  characteristics  of  the 
exercise  are  typically  established  as  permitted  by  available  resources  or 
other  constraints.  The  establishment  of  training  objectives  is  a  byproduct 
of  this  rather  than  the  reverse. 

The  definition  of  training  objectives  and  characteristics  for  SCMs/STEs  is 

complicated  by  a  number  of  factors.  It  requires  expertise  in  military  tactics 

and  doctrine,  own  and  enemy,  plus  additional  knowledge  derived  primarily 

from  combat  experience.  Past  performance  strengths  and  weaknesses  of 
2 

the  C  team  or  type  of  team  must  be  factored  in  to  include  both  team  and 
mission  task  skills.  System  mission  objectives,  especially  new  objectives 
growing  out  of  superteam  needs  (for  example,  the  Airborne  Tactical  Air 
Control  System--ATACS--which  combines  AWACS  and  ABCCC,  or  inter¬ 
service),  play  an  important  role.  Furthermore,  these  objectives  must  be 
put  into  a  realistic  context  characterized  by  a  world- wide  military 
potential  for  TACS  and  AWACS.  The  context  is  defined  by  a  battle  scenario 
supported  by  a  script  and  radar  simulation  or  live  flying,  if  available. 

It  follows  that  as  a  result  of  the  difficulties  of  defining  SCMs/STEs,  they 
are  difficult  to  evaluate  for  validity  and  training  value.  Subject  matter 
expertise  is  crucial  in  all  phases  of  the  definition  and  conduct  of  SCMs/STEs. 
Yet  the  complexities  of  the  planning  task  might  exceed  man's  unaided 
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information  management  abilities.  This  could  account  for  the  lack  of 
systematic  approaches  to  the  definition  of  training  objectives  for  and 
characteristics  of  SCMs/STEs. 

Resolution  of  this  problem  lies  in  the  development  of  information  manage¬ 
ment  tools  which  can  be  used  by  exercise  planners  to  carry  out  their 
task  in  a  more  systematic  manner.  The  primary  source  of  information  for 
development  of  such  tools  is  the  planning  conferences  themselves. 
Observation  of  several  conferences  will  provide  an  understanding  of  how 
SCM/STE  planning  is  currently  done.  This  is  necessary  because  any 
guidelines  or  aids  developed  should  not  depart  significantly  from  present 
methods,  however  loosely  defined.  Otherwise  the  developed  tools  will  be 
used  ineffectively — or  not  at  all.  In  addition,  the  conferences  will  also 
provide  information  on  exercise  objectives  and  constraints,  as  well  as 
information  about  the  characteristics  and  needs  of  potential  users. 

The  constraints  placed  on  the  definition  of  SCMs/STEs  must  be  fully 
appreciated.  These  include  resource  and  policy  constraints  as  well  as 
simulation  capabilities  and  the  like.  The  alternatives  and  options 
available  when  confronted  with  such  constraints  may  be  expandable  or 
may  generate  ideas  for  further  R&D. 

Finally,  observation  of  conferences  is  important  because  the  intended 
users  of  the  guidelines  and  aids  should  participate  in  their  development  to 
as  great  a  degree  as  possible.  They  must  feel  ownership  for  the  tools 
developed  as  a  result  of  this  effort. 
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Contractor  resources  required  for  full  development  of  tools  for  the  definition 

of  SCM/STE  training  objectives  and  characteristics  would  be  on  the  order 

of  four  to  five  person-years  over  a  12  to  15-month  period.  The  expertise 

2 

of  personnel  should  include  AFC  tactics,  doctrine,  and  operations,  in 
addition  to  training  and  simulation.  Staffing  level  would  be  two  senior 
scientists,  one  full  and  the  other  half-time,  and  two  junior  scientists,  both 
half-time,  plus  a  full-time  research  assistant.  The  staffing  level  is  a 
function  of  the  number  and  different  types  of  planning  conferences  which 
need  to  be  studied  in  order  to  ensure  a  general  solution.  At  a  minimum, 
two  planning  conferences  for  Blue  Flag,  TACS,  and  selected  Semi- Automated 
Ground  Environment  (SAGE)  exercises  should  be  observed,  and  one  conference 
of  each  type  should  serve  as  a  test  bed  for  implementation  of  fully- developed 
guidelines.  Attendance  at  these  planning  conferences  will  require  travel 
and  the  support  of  Air  Force  personnel.  Also,  Air  Force  personnel  need 
to  be  available  for  consultation  during  the  entire  period  of  the  development 
effort. 

Given  the  emphasis  on  ownership  that  is  required  for  the  final  product  and 
the  heavy  involvement  for  this  and  for  technical  accuracy  of  Air  Force 
personnel,  an  alternate  strategy  for  development  would  reduce  the 
contractor's  role  significantly  to  that  of  a  consultant,  for  example,  one  to 
two  person-years  over  a  year's  effort.  The  major  development  effort  under 
this  alternative  would  be  accomplished  by  Air  Force  SCM/STE  planning 
personnel. 
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Implementation  resources  required  to  use  the  tools  developed  under  this 
effort  would  be  minimal.  The  conduct  of  planning  conferences  would  be 
affected,  but  they  would  be  carried  out  in  a  more  systematic  fashion. 
Organizational  memory  should  be  enhanced  because  the  loss  of  experienced 
planning  personnel  would  not  result  in  as  protracted  a  learning  curve  for 
replacement  personnel  as  is  now  the  case.  The  impact  of  implementation 

n 

would  be  that  SCMs/STEs  more  clearly  show  a  relationship  between  T 
objectives  and  the  quality  and  quantity  of  events  and  activities  which  occur. 

The  results  of  this  effort  would  certainly  be  appropriate  for  inclusion  in  an 
SRB.  The  solution  addresses  a  problem  that  is  both  general  and  critical. 

The  emphasis  on  the  intended  user  in  development,  especially  under  the 
alternate  strategy  discussed  above,  would  ensure  a  useful  product  that  is 
in  fact  used. 

2 

Develop  Formal  Training  Material  for  Selected  C 
System  Supervisory  Personnel  ™  ~ 

Knowledgeable,  skilled  leaders  are  essential  for  effective  team  performance. 
Team  leaders  usually  occupy  positions  of  supervision  and  management.  The 
skills  required  in  such  positions  are  almost  exclusively  of  the  cognitive, 
team-oriented  variety. 

It  is  often  assumed  that  Air  Force  officers  and  Senior  NCOs  possess 
supervisory  and  management  skill  by  virtue  of  their  experience  or  education 
(Air  Force  courses  are  offered  in  leadership  and  management),  and  this 
is  certainly  warranted  to  some  degree.  However,  the  skills  may  not  be 
fully  developed  when  personnel  turnover  creates  an  opening  which  must  be 
filled.  Furthermore,  and  more  critically,  effective  supervision  and 
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management  within  a  particular  C  system  demands  a  great  deal  of 
knowledge  regarding  the  procedures,  problems,  and  general  solutions 
which  are  applicable  in  various  mission  phases  or  types  of  missions.  Much 
of  the  required  knowledge  for  dealing  with  established  situations  is  presented 
in  positional  handbooks.  However,  there  are  no  standard  references  for 
emergent  situations  and  there  is  no  substitute  for  extensive  observation 
and  experience  in  acquiring  strategies  for  problem  solving  under  emergent 
conditions. 

2 

The  training  of  tactical  air  C  system  supervisors  is  carried  out  on  the  job 
according  to  the  master-apprentice  instructional  model  (see  Volume  I, 
Chapter  in).  There  is  no  formal,  that  is  classroom,  instruction  offered 
and  there  are  few,  if  any,  training  aids  or  materials  available.  These 
shortcomings  were  acknowledged  by  many  survey  respondents.  As  one 
SAGE  student  Senior  Director  aptly  expressed  it:  "Trying  to  learn  how  to 
be  a  Senior  Director  from  a  manual  is  like  trying  to  learn  English  by 
reading  the  dictionary.  "  Consider  in  this  light  that  training  for  SAGE 
personnel  is  the  best  conceived  among  the  systems  surveyed. 

The  need  to  develop  formal  training  for  supervisors  is  recognized  and,  in 
fact,  the  966th  AW  ACS  Training  Squadron  has  plans  to  develop  instructional 
modules  for  the  Senior  Director  and  the  Mission  Crew  Commander.  This, 
however,  has  a  lower  priority  than  the  development  of  instruction  for 
Weapons  Directors  and  Surveillance  personnel.  Similar  plans  do  not  at 
present  exist  at  the  TACS/OTD.  Without  some  structured  formal  training 
new  supervisors  are  unsystematically  exposed  to  their  jobs,  resulting  in 
a  potential  lack  of  standardization  among  team  leaders. 


The  resolution  of  this  problem  requires  accelerated  development  of  formal 
training  for  AWACS  supervisors  and  the  initiation  of  development  for  TACS 
and  JSS  supervisors.  Combat-ready  supervisors  and  training  specialists 
must  collaborate  to  solve  the  problem. 

As  discussed  earlier,  ISD  in  its  present  form  does  not  support  the  develop- 
2 

ment  of  T  .  However,  creative  use  of  the  ISD  type  of  framework  has 
resulted  in  innovative  solutions  to  supervisor  training.  McCutcheon  and 
Brock  (Reference  12),  using  a  job  task  inventory  for  the  Combat  Information 
Center  Watch  Officer  (CICWO)  produced  by  Rundquist  (Reference  13), 
developed  an  audio/visual  conceptualization  program  for  use  in  the  CICWO 
course.  The  results  of  Rundquist' s  task  analysis  indicated  that  monitoring 
and  evaluating  were  two  key  job  actions  required  of  the  CICWO.  Following 
the  development  of  classroom  exercises  permitting  practice  in  monitoring 
and  evaluating  tasks  (Reference  14),  McCutcheon  and  Brock  developed 
two  versions  of  a  "Concepts  of  CIC"  program.  These  programs  showed 
"a  hypothetical  CICWO  detecting  errors,  making  recommendations  to  the 
bridge,  and  so  on.  "  Experimental  use  of  the  programs  indicated  that 
students  who  viewed  them  performed  significantly  better  than  controls 
in  subsequent  tests. 

The  method  of  task  analysis  used  by  Rundquist  (Reference  13)  is  similar 

to  current  techniques.  The  key  seems  to  have  been  in  the  selection  of  a 

training  medium  appropriate  to  cognitive  skill  acquisition.  There  is  no 

2  2 

reason  why  this  approach  cannot  work  in  the  domain  of  Air  Force  C  T  . 
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The  development  effort  suggested  here  does  not  necessarily  have  to  result 

2 

in  a  full-blown  course  for  a  C  system  supervisor.  Although  that  is  a 
desirable  final  goal,  an  achievable  near-term  goal  would  be  the  specification 
and  production  of  training  material  in  the  spirit  of  McCutcheon  and  Brock 
(Reference  12),  which  has  demonstrated  training  effectiveness. 

The  contractor  resources  required  for  full  development  of  exemplary 

training  material  would  be  1.  5  to  2  person-years  for  a  period  of  six  to 

nine  months.  Great  economy  could  be  achieved  if  this  effort  were  integrated 

with  the  development  of  a  team  task  and  skill  taxonomy  and  a  task  analysis 

2  2 

method.  Personnel  qualifications  include  knowledge  of  AFC  T  ,  instructional 

technology,  and  media  production.  Additional  resource  considerations  would 

include  film  or  videotape  production  facilities  and  associated  materials. 

2 

Participation  of  an  experienced  C  system  supervisor  is  a  necessity, 
preferably  one  with  a  training  background.  Depending  upon  the  exact  nature 
of  the  subject  matter,  filming  or  videotaping  a  mock  combat  mission  will 
be  necessary. 

The  implementation  of  the  final  product  would  require  no  substantial  Air 
Force  resources.  Improved  training  and  performance  effectiveness  could 
be  achieved  for  little  cost.  Reduction  in  the  amount  of  on-the-job  training 
(OJT)  required  is  possible;  this  would  have  a  positive  cost  impact. 

The  methods  used  to  develop  the  training  material  would  be  appropriate  for 

2  2 

inclusion  in  an  SRB.  The  probiem  addressed  is  ubiquitous  in  AFC  T  . 
Training  developers  should  be  able  to  readily  use  the  approach  and  methods 
outlined  above. 
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Develop  Supplemental  Training  in  Selected  Skills  for 
Entrants  into  the  Air  Weapons  Controller  Career  Field 

The  demand  for  Air  Weapons  Controllers  (AWCs),  owing  to  low  retention 
rates  and  new  system  deployment,  is  extremely  high,  while  job  candidates 
are  in  short  supply.  Therefore,  in  order  to  maintain  full  manning  levels, 
the  Air  Force  has  not  been  able  to  be  selective  in  choosing  individuals  for 
this  career  field.  The  consequence  is  a  mismatch  between  the  basic  skills 
required  by  the  AWC  job  and  student  qualifications. 

The  gap  created  by  this  mismatch  can  be  closed  by  introducing  certain 
remedial  training  lessons  into  the  AWC  Fundamentals  course  curriculum. 
The  responses  of  instructors  during  the  interviews  indicated  that  remedial 
training  appears  to  be  required  in  mathematical  and  communication  skills. 
Certain  spatial  reasoning  abilities  might  also  be  the  subject  of  training. 

Development  of  supplemental  training  should  proceed  in  an  orderly  fashion 
beginning  with  an  assessment  of  deficiencies  in  mathematical  and 
communication  skills.  This  information  could  be  obtained  from  existing 
records  of  test  scores  (for  example,  the  Air  Force  Officer  Qualifying 
Test--AFOQT)  for  past  and  present  students.  Alternatively,  the  math¬ 
ematical  and  communication  skills  required  by  the  AWC  job  could  be 
analyzed- -it  was  evident  from  the  survey  that  algebra  and  geometry  are 
key  mathematical  skill  areas--and  specific  achievement  tests  could  be 
given  to  the  student  population  and  samples  of  recent  course  graduates 
and  combat-ready  controllers. 


Once  deficiencies  are  pinpointed,  the  appropriate  remedial  training  can  be 
identified.  It  is  likely  that  programmed  instruction  self-study  modules 
already  exist  for  the  skills  required,  so  full-scale  development  of  training 
would  be  unnecessary. 

The  ability  to  reason  spatially,  commonly  assumed  to  be  highly  correlated 
with  mathematical  ability,  is  an  important  attribute  of  good  controllers. 

To  some  degree  this  ability  is  more  of  an  aptitude  than  a  skill  and  so 
training  would  not  be  expected  to  have  a  large  effect.  However,  there  are 
certain  experiences  that  would  appear  to  be  beneficial  in  providing  AWC 
students  with  a  frame  of  reference  for  the  tasks  they  perform.  The 
difficulty  that  many  students  have  in  adopting  a  "God's-eye"  perspective 
on  the  situation  display  (see  Volume  I,  Chapter  III)  may  be  related  more  to 
experience  than  to  innate  aptitude.  After  all,  the  comparable  information 
source  most  familiar  to  AWC  students  is  a  television  or  videogame.  The 
situation  display  represents  a  different  sort  of  environment;  overcoming  the 
initial  problems  some  students  have  in  adopting  a  God's  eye  view  might  be 
resolved  through  a  visual  experience  which  makes  the  difference  clear. 

One  approach  to  resolving  the  problem  would  be  to  make  use  of  videotapes 
or  film  of  the  displays  of  the  Air  Combat  Maneuver  Information  (ACMI) 
system  in  use  at  the  Air  Defense  Weapons  Center  (ADWC).  The  ACMI  system 
is  capable  of  displaying  views  of  aircraft  radar  returns  from  different 
perspectives  including  vertical  and  horizontal.  The  "three-dimensionality" 
inherent  in  the  combination  of  these  views  might  provide  exactly  the 
experience  needed  to  jar  the  AWC  students'  perception.  The  aphorism 
regarding  the  relative  values  of  pictures  and  words  certainly  holds  true  in 
this  instance. 
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The  contractor  resources  necessary  to  fully  develop  the  proposed  solutions 
are  minimal.  The  effort  can  best  be  accomplished  by  the  Air  Training 
Command  (ATC)  assuming  that  the  instructors  have  access  to  expertise 
in  education  technology  and  audio/ visual  media  production.  This  further 
assumes  that  the  instructors  are  authorized  to  pursue  these  suggested 
solutions  and  are  given  the  time  to  accomplish  them. 

Remedial  training  of  the  sort  proposed,  that  is,  programmed  instruction 
self-study,  will  not  be  costly  to  the  Air  Force.  Increases  in  training 
effectiveness  and  in  graduate  quality  can  be  achieved  for  a  minimal 
investment  in  materials. 

This  proposed  solution  is  not  appropriate  for  inclusion  in  an  SRB.  The 
emphasis  should  be  on  positive  action  to  identify  the  specific  needs  and 
to  implement  the  required  remedial  training.  Supporting  background 
material  on  skills,  entry  requirements,  and  their  relationship  to  learning 
and  remediation  should  also  be  provided. 

Develop  Formal  Training  for  Interceptor  Pilot  Simulators 

Interceptor  pilot  simulators  (IPSs)  are  those  personnel  who  simulate 
pilots  when  necessary  during  institutional  or  operations  training  of  AWCs. 
Typically  in  initial  and  initial  transition  training,  the  IPS  is  an  Aerospace 
Control  and  Warning  Systems  Operator  (ACWSO),  surveillance  technician, 
plotter,  teller,  recorder,  etc.,  who  has  not  achieved  combat-ready 
status.  In  transition,  upgrade,  and  continuation  training  the  IPSs  are 
experienced  ACWSO  personnel  or  AWC  instructors. 
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The  task  of  an  IPS  is  to  provide  the  necessary  dialogue  for  AWC-pilot 
subteam  training.  Some  of  the  dialogue  can  be  anticipated  and  put  into 
a  script.  However,  ACWSOs  have  had  no  piloting  experience  and  little  if 
any  flight  experience,  and  beyond  instructions  in  equipment  operation  and 
vocabulary  they  have  had  no  training.  Consequently,  the  training  experience 
for  the  AWC  is  suboptimal,  especially  when  the  situation  is  not  scripted. 

This  problem  is  highly  noticeable  in  initial  and  initial  transition  training. 

Some  motivated  IPSs  were  observed  during  site  visits  to  take  their  temporary 
job  seriously.  Some,  for  example,  adopted  three  different  voices  to 
represent  three  different  pilots.  Comments  by  students  and  instructors 
indicated  that  this  practice  improved  the  training  value  of  exercises.  These 
self-taught  IPSs  need  to  be  more  widely  emulated. 

A  reasonable  approach  to  this  problem  would  be  to  develop  a  training 
package  aimed  at  raising  the  level  of  skill  and  motivation  of  IPSs.  It  may 
well  be  that  a  better  understanding  of  their  role  and  impact  would  improve 
their  motivation  sufficiently.  This  development  could  be  carried  out  fairly 
readily  on  an  in-house,  that  is.  Air  Force,  basis.  Contractor  support  could 
be  obtained  to  aid  the  specification  of  IPS  tasks  and  training  requirements. 
Such  support  could  be  obtained  on  a  level  of  effort  basis  not  to  exceed  one- 
half  person-year  of  a  junior  scientist. 

Development  of  an  IPS  training  package  would,  in  our  opinion,  improve 
training  effectiveness  by  increasing  the  realism  of  simulation  exercises. 
Preparation  of  the  package  would  entail  only  a  small  investment.  If  a 
self-study  approach  is  adopted,  the  Air  Force  implementation  cost  would 
be  negligible. 


The  results  of  this  effort  would  be  appropriate  for  inclusion  in  an  SRB. 

This  would  be  primarily  in  the  nature  of  the  identification  of  a  training 
2 

resource  for  the  T  program  manager,  not  the  materials  themselves, 
which  could  be  made  obtainable  through  the  Air  Force  documentation 
systems. 

T2  PROGRAM  EVALUATION 

2 

Improve  Validity  of  T  Program  Evaluation  Procedures 


Current  questionnaire  approaches  to  initial  and  initial  transition  training 
program  evaluation  are  of  suspect  validity.  Comments  and  criticisms 
from  operations  personnel  are  often  taken  with  a  "grain  of  salt"  because 
they  have  dwelled  on  the  fact  that  students  lack  preparation  in  procedures 
and  techniques  which  are  required  or  taught  locally.  Consequently,  the 
institutional  personnel  feel  that  the  operational  personnel  are  not  sensitive 
to  their  problems  and  training  requirements.  Improvements  in  the  quality 
and  quantity  of  the  questionnaire  data  received  by  the  schools  from  the 
field  could  enhance  program  evaluation  and  modification.  A  better- articulated 
purpose  and  structure  of  the  questionnaire  technique  would  enable  the 
field  personnel  to  address  issues  related  to  job  requirements,  including 
team-oriented  skills.  Furthermore,  the  school  could  use  the  questionnaires 
to  provide  more  quantitative  data  regarding  the  success  of  the  students 
they  trained,  such  as  the  time  to  upgrade  to  combat-ready  status. 


The  state  of  the  art  in  opinion  surveys  has  made  significant  advances  in  the 
last  several  years;  for  example,  poll-taking.  Program  evaluation  has 
become  a  scientific  endeavor,  although  the  construction  of  useful  question¬ 
naires  remains  something  of  an  art.  A  critique  of  the  questionnaires 
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currently  in  use  in  AFC  T  should  be  undertaken  to  determine  in  which 
areas  additional  information  is  needed  or  in  which  format  the  probability 
of  obtaining  useful,  valid  information  is  maximized.  A  revised  question¬ 
naire  would  then  be  developed  and  administered  by  the  schools  to  a  sample 
of  operational  sites.  It  would  be  evaluated  in  terms  of  coverage  and  the 
reactions  of  users  and  respondents. 

This  development  effort  would  require  a  six-  to  nine-month  period  of 
performance.  Personnel  would  consist  of  a  half-time  senior  scientist, 
a  full-time  junior  scientist,  and  a  full-time  research  aide.  Requisite 
skills  are  in  the  areas  of  program  evaluation,  survey  development  and 
implementation,  and  statistical  methods. 

Air  Force  implementation  costs  associated  with  the  development  of  an 
improved  questionnaire  would  be  negligible  except  for  personnel  time. 
However,  depending  upon  the  results  there  will,  of  course,  be  costs 
associated  with  program  modification. 

Perhaps  the  biggest  benefit  of  undertaking  this  effort  wouid  be  to  close  the 
loop  between  the  field  and  school  environment  by  reducing  the  mismatch 
between  training  and  operational  needs. 

The  resultant  questionnaire  could  be  included  in  an  SRB.  The  questionnaire 

2  2 

should  be  program-independent,  and  AFC  T  program  managers  should 
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be  able  to  use  it  within  guidelines  provided  in  an  SRB.  It  thus  could  be  a 

2  2 

universal  tool  in  program  evaluation  for  AFC  T  . 

Develop  Procedure  for  Compiling  After- Action 
Reports  into  a  Training  Exercise  Data  Base 

The  after-action  report  (AAR),  which  characterizes  team  and  system 

performance  as  observed  during  STEs/SCMs,  is  not  used  to  its  fullest 
2 

advantage  in  T  program  evaluation.  The  AAR  currently  is  used  only 
as  a  "one-shot"  mechanism.  A  compilation  of  AARs  with  analysis  and 
synthesis  would  provide  data  descriptive  of  positive  and  negative  aspects  of 
team  performance.  The  data  could  be  used  in  numerous  ways  including  the 
following: 

1.  Improve  the  AAR  format  so  that  it  more  directly  addresses 
feedback  to  teams  about  team-oriented  skills. 

2.  Identify  issues  requiring  improved  training  or  modified  or 
new  operating  procedures. 

3.  Identify  issues  of  system  design  which  affect  team  performance. 

4.  Provide  a  reference  for  new  team  members,  especially 
supervisors. 

5.  Provide  a  rich  source  for  specific  research  issues. 


The  procedure  for  compiling  the  AARs  should  consist  of  an  analysis 
framework  like  that  depicted  in  Figure  2  and  a  synthesis  framework  like 
that  depicted  in  Figure  3. 
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Figure  2.  Analysis  Framework  for  After-Action  Reports 
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Figure  3.  Synthesis  Framework  for  Compiled  AAR  Data 


Figure  2  shows  that  the  data  from  AARs  can  be  sorted  into  task/skill 
categories  applicable  to  different  performance  contexts.  The  performance 
contexts  have  been  described  in  Chapter  I,  Volume  I.  Individual  and  subteam 
contexts  have  been  collapsed  into  a  preteam  context  for  this  use. 

The  mission  tasks  are  operationally  defined  by  the  system  and  the  particular 
STE/SCM  scenario.  The  team-oriented  tasks  include  the  following: 

•  Communications,  for  example,  discipline 

•  Adaptability  to  emergent  events 

•  Anticipation,  for  example,  consequences  of  own  action  of 
information  needs  of  others 

•  Adjustment  to  workload  modification 

•  Problem  solving/decision-making 

Figure  3  shows  the  framework  which  would  be  used  to  synthesize  the 
compiled  data.  There  are  three  different  elements  which  may  be  character¬ 
ized  in  terms  of  strengths,  deficiencies,  and  the  operational  consequences. 
These  elements  are  human,  system  (that  is,  hardware  and  software), 
and  operating  procedures  which  vary  from  well-defined  (established)  to 
ill-  or  non-defined  (emergent).  The  operational  consequences  of 
deficiencies  will  lead  to  recommendations  regarding  training  needs  or 
corrective  actions  involving,  perhaps,  system  design  (future  systems,  for 
example)  or  the  development  or  modification  of  operating  procedures. 

This  application  effort  would  require  one  half-time  senior  psychologist, 
one  full-time  junior  psychologist,  and  one  half-time  research  associate. 


The  period  of  performance  would  be  six  to  nine  months.  A  prerequisite  for 

2  2  2 

successful  performance  is  intimate  knowledge  of  AFC  T  ,  C  systems,  and 
2 

T  technology,  such  as  it  is. 

Development  of  the  compilation  procedure  would  require  the  cooperation 

of  a  stan/evai  section  of  the  target  system  (a  CRC  would  be  ideal). 

Implementation  of  the  developed  procedure  would  require  participation 

by  stan/eval  personnel  and  T  program  managers.  The  task  of  compiling 

past  AARs  would  add  to  the  already  high  workload  of  these  personnel,  but  it 

2 

would  be  justified  if  T  effectiveness  were  increased.  If  the  result  of  this 
effort  lent  itself  to  a  revised  procedure  for  generating  AARs  (for  example, 
feedback-oriented  reports),  this  could  have  a  large  positive  impact  for  a 
small  imposition  on  current  workload. 

The  procedures  developed  under  this  effort  would  be  appropriate  for 

inclusion  in  an  SRB.  The  compilation  procedure  would,  however,  be 

difficult  to  use  and  would  require  some  special  expertise  or  analytic  skills 
2  2 

which  AFC  T  personnel  might  not  possess.  It  is  therefore  more  likely  that 
a  revised  AAR  generating  procedure  could  be  successfully  implemented  in 
an  SRB  format  because  it  would  extend  a  current  procedure  or  at  least  be 
related  to  it. 
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PERSONNEL  POLICY 


Conduct  Attitude  Surveys  to  Pinpoint  Sources  of 
Discontent  in  C^  Career  Fields  and  Recommend  Changes 

2 

Low  retention  rates  of  C  system  personnel  are  a  serious  problem  owing 
to  the  importance  of  experienced  individuals  to  the  development, 
implementation,  and  management  of  successful  institutional  and  operational 
training. 

Policies  affecting  pay,  work  conditions,  assignments,  and  career 
opportunities  need  to  be  examined  closely.  Useful  data  for  such  policy 
review  could  be  obtained  through  an  attitude  survey.  The  state  of  the  art 
of  surveying  employee  attitudes  has  advanced  significantly  in  the  last 
several  years.  A  concerted  survey  effort  uging  current  techniques  would 
provide  a  necessary  data  base  for  policy  decisions,  plus  indicate  to  the  C 2 
community  that  the  "employer"  cares.  The  need  to  train  new  personnel 
would  be  cut  dramatically  if  retention  of  experienced  individuals  could  be 
enhanced. 

The  survey  development  and  conduct  would  require  on  the  order  of  six  to 
nine  months.  A  half-time  senior  scientist,  a  full-time  junior  scientist,  and 
full-time  research  associate  would  be  required.  Expertise  in  industrial 
and  organizational  psychology,  organization,  and  human  resources 
development  is  required.  In  terms  of  Air  Force  resources,  the  cost  of 
implementing  a  developed  survey  would  be  negligible.  The  costs  of  policy 
decisions  resulting  from  recommendations  are  unknown. 
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The  survey  instrument  could  be  included  in  an  SRB.  It  could  be  used  in 
other  or  related  personnel  areas  as  well. 


Develop  Instructor  Resource  Package 
for  Operational  Unit  Instructors 

Most  instructors  in  operations  training  programs  have  had  little,  if  any, 
training  in  how  to  train.  They  are  chosen  primarily  on  the  basis  of  their 
positional  proficiency.  More  specifically,  there  exist  no  references  for 
operations  training  program  instructors  to  aid  them  in  their  job,  the  most 
important  aspect  of  which  is  to  ensure  that  individuals  can  function 
effectively  in  a  team  environment. 

The  Air  Training  Command  runs  a  four-week  instructor  training  course 
which  could  serve  as  the  basis  for  instructor  training  in  operational  units. 
This  course  could  be  condensed  and  taught  by  a  mobile  training  team  or  it 
could  be  offered  strictly  as  a  correspondence  course.  In  any  event,  such 
a  course  should  be  required  for  all  operational  unit  instructors. 

2 

The  content  of  the  course  should  be  modified  or  expanded  to  include  T 

considerations.  Instructor  awareness  of  distinctions  among  the  different 
2  2 

AFC  T  performance  contexts,  for  example,  could  allow  feedback  to  be 
delivered  more  appropriately.  A  description  and  discussion  of  team- 
oriented  skills  could  focus  instructor  attention  on  those  skills  and  allow 
more  pertinent  and  timely  feedback  to  be  provided. 

The  development  of  an  operational  unit  instructor  training  package  would 
require  one  three-quarter  time  senior  psychologist,  one  three-quarter 


35 


time  junior  psychologist,  and  one  half-time  research  assistant.  The  effort 

could  be  completed  in  9  to  12  months.  A  successful  effort  requires 
2  2 

knowledge  of  AFC  T  and  training/ instructional  techniques.  Economy  in 
the  contractor  portion  of  this  effort  could  be  realized  if  ATC  materials 
and  personnel  were  directly  involved. 

A  requirement  for  operational  unit  instructors  to  undergo  training  would 
add  to  their  already  heavy  workload.  However,  the  expected  enhancement 
in  training  effectiveness  would  trade  off  well  against  this  increased 
commitment. 

If  the  training  package  were  in  a  self-study  format  it  would  be  ideal  for 
inclusion  in  an  SRB.  The  material  would  be  directly  usable  by  the 
instructors  and  should  positively  impact  their  ability  to  train  effectively. 
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CHAPTER  IH 

RECOMMENDATIONS  FOR  APPLICATION  OF 
CURRENT  TECHNOLOGY 

The  preceding  chapter  discussed  11  application  ideas  which  could  lead  to 

immediate  improvement  of  the  development,  implementation,  or  evaluation 
2  2 

of  AFC  T  .  The  purpose  of  this  chapter  is  first  to  select  one  application 
topic  for  pursuit  during  the  second  phase  of  this  contract  effort  and  second 
to  describe  the  proposed  Phase  II  effort  in  detail. 

SELECTION  OF  APPLICATION  TOPIC  FOR  PHASE  II 

All  1 1  application  ideas  should  be  undertaken  in  one  form  or  another 

2  2 

because  they  address  existing  issues  and  problem  areas  in  AFC  T  . 
However,  in  the  context  of  a  Phase  II  effort  the  criteria  of  feasibility, 
utility,  usability,  probability  of  success,  and  practical  payoff  must  be 
weighed  in  order  to  determine  priorities.  In  this  section  these  criteria  are 
defined  and  each  application  idea  is  evaluated  on  each  criterion  so  as  to 
produce  an  overall  priority  ranking. 

Feasibility 

In  the  context  of  a  Phase  II  effort  feasibility  means  availability  of  time  and 
labor.  Phase  II  must  be  conducted  in  nine  months.  It  will  involve  a 
fifth-time  senior  psychologist,  a  two-thirds-time  junior  psychologist,  and 
a  two-thirds-time  research  assistant.  Feasibility  also  considers  the  skills 
available  as  compared  to  those  required. 
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Utility 


Utility  refers  to  the  pertinence  of  the  effort  to  the  conduct  of  Air  Force 
training  and  operations. 

Usability 

The  product  of  the  effort  must  be  usable  by  Air  Force  training  personnel. 
Probability  of  Success 

The  exis*  'e  of  data,  techniques,  and  other  resources  will  determine 
probability  of  success.  Technical  difficulty  of  achieving  the  application 
objective  is  also  factored  in. 

Practical  Payoff 

If  the  effort  is  successful,  to  what  degree  will  it  positively  impact  the 

2  2 

effectiveness  or  efficiency  of  AFC  T  ? 

Priority  for  Phase  II 


Table  4  summarizes  the  results  of  the  consideration  of  the  above  criteria. 
Each  criterion  was  judged  on  a  five-point  scale  with  1  being  low  and  5  being 
high.  The  total  of  the  ratings  is  simply  a  non-weighted  sum  and  the  rank  is 
from  highest  total  to  lowest. 
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TABLE  4.  PRIORITIZATION  OF  APPLICATION  TOPICS 


2 

As  can  be  seen  from  the  table,  the  development  of  training  for  C  team 
supervisors  (Topic  5)  has  the  highest  priority  for  the  Phase  II  effort  as 
defined  by  the  constellation  of  evaluation  criteria. 

DEVELOPMENT  OF  C2  TEAM  SUPERVISOR  TRAINING 

In  order  to  more  fully  define  this  Phase  II  topic  the  application  context, 
objectives,  approach,  products,  and  required  resources  are  reviewed 
and  expanded  on  in  this  section. 


Application  Context 

The  Airborne  Warning  and  Control  System  (AWACS)  is  recommended  as 

the  target  application  context.  Furthermore,  the  Mission  Crew  Commander 

2 

(MCC)  position  in  AWACS  is  recommended  as  the  C  team  supervisor  for 
whom  to  develop  training  materials. 

AWACS  provides  capabilities  in  surveillance,  warning,  and  control  that 

were  heretofore  unattainable.  The  battle  commander  who  is  supported  by 

an  AWACS  has  dramatically  increased  chances  of  overall  mission  success. 

Furthermore,  AWACS  is  a  microcosm  of  larger  (in  terms  of  personnel), 

2 

more  complex  C  systems  and  has  a  limited  number  of  links  to  other 
components.  It  is  therefore  somewhat  easier  as  compared  to  the  T ACC  or 
CRC  to  specify  input-output  and  network  relationships  among  the  AWACS 
mission  crew  and  between  the  AWACS  and  other  components. 


These  considerations  mean  that  AWACS  provides  a  high-payoff,  tractable 
application  context  for  the  Phase  II  effort.  In  addition,  because  there 
is  currently  no  formal  training  for  the  MCC,  the  development  of  training 
materials  will  fill  an  immediate  need. 

Objective 

The  objective  of  this  effort  is  to  demonstrate  the  feasibility  of  developing 

2 

training  material  for  a  C  team  supervisor  and  evaluate  that  material  for 
appropriateness  and  training  value.  A  parallel  objective  is  to  document 
the  procedure  whereby  the  training  material  is  produced  and,  also,  the 
evaluation  results. 


Approach 

The  approach  will  follow  that  used  by  McCutcheon  and  Brock  (Reference 
12)  in  their  highly  successful  effort  to  develop  training  for  the  Combat 
Information  Center  Watch  Officer  (CICWO).  The  steps  are  as  follows 
(these  steps  are  presented  in  an  annotated  outline  in  Chapter  IV  of  this 
volume): 

1.  Identify  critical  job  tasks 

2.  Identify  skill  categories 

3.  Establish  skill  acquisition  guidelines 

4.  Define  exercise /experience  characteristics 

5.  Select  training  media 


6.  Develop  set  of  scripted  exercises 

7.  Produce  training  materials 

8.  Evaluate  materials 

The  present  effort  will  focus  on  those  job  tasks  requiring  team-oriented 
decision-making  and  managerial  skills  in  non-standard  operational  situations 
or  situations  for  which  operating  procedures  have  not  yet  been  developed. 

The  identification  of  these  types  of  job  tasks  will  require  on-site  data 
collection  and  the  assistance  of  a  combat-ready  MCC  who  has  a  background 
in  training.  The  data  collection  effort  will  involve  observation  of  live  and 
simulated  missions  and  interviews  with  combat-ready  MCCs.  A  minimum 
of  three  live  (flying)  simulated  combat  missions  will  be  observed  with 
three  different  MCCs.  These  missions  will  be  flown  by  operationally  ready 
crews.  If  appropriate,  that  is,  if  an  MCC  participates,  a  minimum  of  two 
missions  in  the  AWACS  simulation  facility  will  be  observed.  All  observed 
missions  will  be  followed  as  soon  as  practical  by  interviews  with  the 
MCCs  involved.  Furthermore,  interviews  with  several  additional  MCCs 
will  be  conducted.  In  total,  six  to  eight  MCCs  will  be  observed  and  inter¬ 
viewed  and  from  this  data  base  critical  job  tasks  and  events  will  be 
identified.  It  is  estimated  that  the  interviews  will  each  take  two  hours  to 
complete. 

These  data  will  be  reviewed  with  the  training- knowledgeable  MCC;  and  based 
upon  Air  Force  approval,  the  remainder  of  the  steps  outlined  above  will  be 
carried  out.  No  direct  Air  Force  support  is  required  again  until  step  6. 
After  step  5,  but  before  the  scripted  Human  Resources  Laboratory  and 
AWACS  exercises  are  produced,  Air  Force  approval  of  the  intermediate 


results  will  be  obtained.  In  particular,  the  exercise  characteristics  and 
selected  media  will  be  examined  for  relevance  and  validity.  Finally,  the 
scripted  exercises  produced  in  step  6  will  be  reviewed  and  approved  before 
production. 

The  evaluation  of  the  training  materials  produced  will  require  on-site  data 
collection  consisting  of  demonstration  and  use  of  the  materials.  Both 
student  and  combat- ready  MCCs  will  participate  in  the  evaluation,  possibly 
in  a  group  setting.  The  primary  evaluative  tools  will  be  observation  of  use 
and  a  questionnaire.  It  is  estimated  that  each  participant  in  the  evaluation 
will  be  involved  for  two  hours. 

Product 


It  is  envisioned  that  the  product  of  this  effort  will  consist  of  audio  and/or 
visual  programs  supported  by  text.  The  programs  will  illustrate  the  team- 
oriented  supervisory /decision  skills  required  of  the  MCC  during  especially 
critical  mission  segments.  The  text  and/or  other  media  will  contain 
descriptive  information  including  background  and  a  discussion  of  alternate 
actions  and  possible  consequences.  The  materials  will  illustrate  correct 
as  well  as  incorrect  decision/actions  and  their  consequences. 

Resources  Required 


Three  on-site  visits  to  Tinker  Air  Force  Base  will  be  required  during  this 
effort,  as  shown  below. 


Visit 

Length  (days) 

Purpose 

Period 

1 

7-10 

Data  Collection 

15  May  to  15  June  1981 

2 

1-2 

Briefing  on 
Intermediate 

15  June  to  30  June  1981 

Results 

3 

2-3 

Demonstration/ 

Evaluation 

15  August  to  31  August  1981 

A  subject  matter  expert  (training-knowledgeable  MCC)  will  be  required  for 
occasional  telephone  consultation  throughout  the  period  of  performance. 

The  production  of  audio  and/or  visual  materials  from  contractor-written 
scripts/text  will  be  accomplished  by  Air  Force  audio/visual  experts.  The 
finished  audio  and/or  visual  presentation  is  expected  to  be  no  more  than 
one  hour  in  length.  The  production  effort  will  take  two  to  four  weeks  and 
will  involve  approximately  three  person-months. 

Benefit 

Knowledgeable,  skilled  leaders  are  essential  for  effective  team  performance. 

2 

Formal  training  materials  for  C  team  supervisors  will  enhance  individual 
knowledge  and  skill,  plus  facilitate  the  standardization  of  training 
experiences.  Providing  the  AWACS  MCC  a  structured,  systematic  training 
experience  outside  of  the  job  performance  context  will  potentially  reduce 
costs  and  increase  operational  readiness. 
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CHAPTER  IV 


ANNOTATED  OUTLINE  OF  PROCEDURE  FOR 
DEVELOPING  C2  TEAM  SUPERVISOR  TRAINING  MATERIALS 


This  chapter  presents  an  annotated  outline  of  the  procedure  which  will  be 

2 

used  to  develop  training  materials  for  C  team  supervisors. 

I.  INTRODUCTION 

A.  Describe  Purpose  of  the  Procedure 

B.  Discuss  Philosophy  Behind  the  Procedure 

1.  Systematic  development  of  training 

2.  Based  on  ISD-like  sequence 

C.  Give  Overview  of  Procedure 

This  overview  will  be  presented  in  flow  chart  format. 

II.  PROCEDURE 

A.  Develop  Job  Task  List 

1.  The  input  to  this  step  will  be  any  existing  data;  for 
example,  a  job  task  inventory,  procedural  handbook; 
or,  if  data  do  not  exist,  the  task  list  will  be  developed 
through  observations  and  interviews. 

2.  The  output  of  this  step  is  a  duty  description  that  defines 
the  actions,  conditions,  and  desirable  outcomes  of  job 
performance. 


B.  Identify  Skill  Categories 


The  job  tasks  must  be  mapped  into  skill  categories.  The 
emphasis  will  be  on  identifying  cognitive,  team-oriented 
skills.  A  partial  list  of  these  skills  includes  the  following: 


•  Monitor 

•  Evaluate 

•  Recognize  problems 

•  Make  decisions 

•  Coordinate 


•  Communicate 

•  Resolve  conflicts 

•  Adapt  to  emergent 
conditions 

•  Etc. 


C.  Establish  Skill  Acquisition  Guidelines 

In  this  step  the  guidelines  for  effective  acquisition  of  the 
identified  skills  are  established.  These  include: 


1.  Stimulus  conditions 

2.  Complexity/ difficulty  criteria 

3.  Setting 

4.  Feedback  characteristics 

D.  Define  Experience/ Exercise  Characteristics 

Based  on  the  skill  acquisition  guidelines,  the  training 
experiences  are  fully  defined  and  described. 

E.  Select  Appropriate  Media 

Based  on  the  experience /exercise  characteristics,  appropriate 
training  media  are  selected.  The  demonstrated  success  of 
the  observational/ experiential  learning  approach  of  McCutcheon 
and  Brock  (Reference  12)  will  weigh  heavily  in  media 
selection. 
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F.  Develop  Set  of  Scripted  Exercises 

This  is  the  final  step  before  production.  Review  of  the 
scripts  by  subject  matter  experts  is  essential.  The  exercises 
will  illustrate  correct  techniques  and  errors  as  examples. 
Errors  of  omission,  commission,  and  timing  will  be 
illustrated  along  with  their  operational  consequences. 
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